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ABSTRACT 

The purpose of this preliminary study was to gain 
insight into what is being measured by the 1974 Washington State 
Board of Pharmacy Licensure Examination , and to explore the 
possibility of using it as a model of tests to evaluate experiential 
learning. Subjects were 78 examinees from University of Nashington 
and Washington State University. Subjects* grade-point in academic 
subjects in their educational background correlated with the 
"theoretical" subscores of the test. Thus, it appeared that a 
candidate's ability to perform in the test situation may influence 
how well he performs on the licensure examination. Conversely, the 
scores on neither part of the "practical" portion correlated with the 
grade-point average. Whether a subject had completed a rotating 
externship was not found to affect performance on the various parts 
of the examination; however the small numbers of former externs and 
the fact that their characteristics were not greatly different from 
other candidates made any conclusions tentative. (Author) 
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Abstract 



Xbe purpose of this preliminary stuOy was to efidn Insiglit into 
Khat is being aeatorsd by the X^rjk W^ington State Board of Elkan&acy 
Uneeasure Ebeaaination, and to aaq^lore the possibiU'^ of using it as 
a aodel of tests to evaluate o^ericmtial learning. Subjects vere 78 
exaninees trm University of Washington and t^Mhington State 
tJtaiversity. Subject's cprade-point average in acadosiic mibjects in 
his/her educational background correlated with the ''theoretical" 
subsoores of the test. Thus, it appeared that a candidate's ability 
to perfom in the test situation may influence hoir well he/riie 
perfozBs on the licensure exanination. Conversely, the seores on 
neither part of the "practical" portion correlated nith the 
grade-point average. Whether a subject had eoo^leted a rotating 
extetnship was not found to affect perfoxaance on the various parts 
of the escaainationt however the small numbers of foxner externa and 
the UiGt that their diaracteristies vere not greatly differsnt fron 
other candidates made any conclusions tentative. 
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Riaxioacy Stdte Board S:':BminatioQ8 and Practical Experience 
Ifathaa A. H%U Stanford M. IHial 

Gerald M. Qilli&ore 

As our edccatlon of people for pliazmacy practice increasingly becomes 
concerned vrl^h the process taVing place outside of the school classroom or 
laX>oratory, we find ourr;elves in need of good evaluative tools. These 
tools mst be dr;e loped arl t^utci before educators effectively can plan 
their efforts btA before Eoards of Hiarmacy effectively can fulfill their 
certification role. 

The state br>ard e^avdnaticn has served traditionally in the 
certificatlrn process, but as one of us (1) pointed out 12 years ago> It 
had evolved from a teat of inforaal, individiTa! ized education often to 
become only a retest of academic subject irntter. The lUVBP Standardised 
Blue Ribbon Exaaination has helped greatly in test design and adminlstra* 
tion, even if the input fron> classroom educators continues to overshadow 
that of the practitioners. The tfsst is unquestionably superior to 
individual Board aemberr* efforts end does have a more realistic relation 
to practice than in the past. 

We in educatici:! recently have teem involved with extemships, 
clinical cl'?:^!:s'iips, rjid other forms of c:{ppriential education* We find 
that evaluative methods suitable lor the school- cloister are not satis- 
factory for this type of education. We find we are faced with problems 
similar to those facing the Boards: in trying to assess the eontributioii 
that this type of elucallcn maltes to professional developovuit. Oir 
eactemships ere frequently just structured internships, oeourring during 
the schooling process. 

The authors believe that tt'i state board examination nay be useful 
in develijping evaluative instruc^^nts useful to both the Boards and the 
Colleges. We recognize that the licensure eicsmination is designed to 

♦ Professor of Flicmacy, Ttoiversity of Washington. 
** Chairman, Washington State Board of I^axmacy. 

♦ Assooia':;e Dii-actor, Baucaticnal Assessment Center, ttoiversi^ of 

Washington. 
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measure an adequate level of competence 0Qly> while Colleges nay wish to 
differentiate several levels of knowledge and activity. Also, it does not 
cover the complete range of educational ea^teriences in esctemshlps. Never- 
theless, it is a start. With the cooperative effort of Boards and 
Colleges as demonstrated by this preliminary study, hopefully we can 
inaigjbt into what is being measured by the licensure examination and the 
possibility of using it as a model of tests to evaluate experiential 
learning. 

This study is prelininaxy . That point must be oaphasized. Any benefit 
to examining groups, educators, and candidates will cane through sophisti- 
cation of techniques and careful evaluations during the development process, 
rather than from the results of this single stiidy. 
Description of the Study 

The subjects were candidates who took the 197^ Washington State Board 
of Pharmacy Ucenaure Sxamination. Graduates who had cosq^leted the Uhi- 
/ersity of Washington (UW) extemship program were the te«t 8ub;)ects. 
Other graduates of UW or Washington State Uhiversity (WSU) weie the control 
subjects. The prime object of the study was to determine if differences 
could be detected between the performances of the test group and the control 
group on the examination. The test and control groups were of different 
sizes, emd persons in each group had undergone a variety of educational 
experiences, many of them experiential, during their school years. Out 
results are therefore only tentative due to lacB: of maibers and homo- 
geneity of sample treatment. Ideally, both groups would take a test prior 
to their experience and another after the ejq?erience, to give more specific 
measure of Itnowledge gained. 
Procedures 

The Washington State Board of Pharmacy regulations require 1500 hours 
of internship credit; of these 500 hours are given for graduation from WSU 
or UW, so for these candidates the remaining 1000 hours are to be satisfied. 
Three hundred hours are required to be served after graduation and 700 
hours r^y be served prior to graduation. Of the 700 hours, a maxlmiim of 
500 hours credit may be givoi for experiential courses taken in school* 
The control group and sane of the extern test group had satisfied the 
internship criteria, up to con^leticm of postgradtiate eacperience (TOO hours). 



The rotating extemship program has been explained to this group 
previously In 1974 there were X8 students t<ho coi&pXeted the program 

(in cossBunity, hospital, and on- floor clinical pharmacy service) and were 
admitted to the examination. For some of these graduates, the time they 
had spent in the full-tlJ&e extemship was counted for internship erediti 
so that they could meet the TOO hour minimum necessary to take the 
examination. 

At the 1974 examination, all candidates filled out a supplementary 
personal data questionnaire which Included the qiaestions shown r", Table 1. 



BEST on AWULABU Kibie i 

Supplmeatary Personal Data 



X. What bets been your total phaxnacy related experience to d&te? 
Choices: 700-999 hours, 1000-1^99 hours, 1^00 hours or more 

2. How tsuch j^pprovable ea{;perience have you had? 

Choices: 700-999 hours, 1000-1^^99 hours, 1500 hours or more 

3. Which of the following describes your use of the 500 hours credit 
allowed for courses you took in school? 

Choices: None, 1-50 hours, 51-100 hours, 101-200 hours, 201-300 hours 

k» Did you complete an extemship program? 
Choices: Yes, K6 

5. Please give your latest cumulative grade point average. 

6. From what pharmacy school did you graduate? 
Choices: UW, WSU, Other 

7. Have you completed the post-graduate experience part of the 
internship requirement? 

Choices: Yes, No. 



After the examination was graded, the Board of Pharmacy jEUmlshed 
the following percoitage scores (unadjusted) frcsn the NABP standardized 
examination administered to all candidates: Chemistry, Mathematics, 
Hjaimacy, Ehaxmacology, Practice of iliaroacy (Written Practical), and 
Errors and Ctaissions. For this study, none of the candidates were identi- 
fied by name but the extemship graduates were Identified by special 
numbers. There were 18 extern candidates, 60 non-extern candidates. 

Pata from the qjaestionnedres and the examination scores were 
anaOyzed in two «ays. First, variables were intercorrelated, where ap- 
propriate, using the Pearson Product-Moment method. The magnitudes of the 
resulting correlation coefficients were tested for significance. Secondly 
a t-test of differences between mean^ was used to assess the significance 
of the differences between scores of UW and WSU graduates and between 
extern and non-extern groups. 
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Results and Discussion 

General Correlations 

Since written tests often measure a person's ability to respond in a 
siJBilar way to a test situation, it is interesting to examine correlations 
aaiong items that reflect ability as well as subject isnowledge. The corre- 
lations among ttm^ that reflect ability as well as subject knowledge* 
!l!he correlations axuonQ the scores of the *> ' censure examination, grade 
point average, and pharmacy experiences presented in Table 2. Those 
correlations which are significantly different from sero at the 0.0$ level 
are marked with an asterisk. 



Table 2 

Pearson Product-I^ent Correlations 



Variable 






• 


• 

8 


t» 


ft 




licensing Sxamination 


• 

3 
6 


i 


1 


1 


Srroi 




















Chemistry 
















Mathematics 
















Hsarmacy 


0.i*5» 


0.59* 












£%armacology 




0,k6* 


0.55* 










Errors/emissions 


0,0k 


0.21* 


0.21* 


O.lif 








Written Practical 


0.58* 


0.55* 


0.i*0* 


0.51* 


O^kJ* 






Experience 
















Total Vb&xmcy 


0.01 


0.08 


0.09 


0.14 


0.2U* 


0.52* 




Approvable 


0.01 


0.15 


.0.01 


0.19 


0.28* 


0.S5* 


0.50* 


Academic 
















Cumulative 


0.57* 


0.59* 


0.56* 


O.kO* 


0.35 


0.19 


.0.07 


Gr. Pt. Av. 
















«p<0.05 
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one cannot avoid speculation about the meanings of the findings* 
Sl0^flcant correlations were evident among all of the parts of the examina- 
tion traditionally referred to as "theoretical." Also, the candidate's 
grade-point average in academic subjects In his/her educational badtgztMind 
correlated with these same "theoretical" parts. Thus> it appears that a 
candidate's ability to perform in the test situation may Influence hovr well 
he/she performs on the licensure examination. 

Conversely, the scores on neither pf^rt of the "practlcaa" portion 
(Brrors and emissions. Written Practical) correlated with the grade-point 
average. This correlation failure may indicate that the "practical" 
section Is measuring a different ability than just that needed to pass 
examinations. The correlation between the t^*o parts of the "practical" 
section indicates that both the Errors and emission test and tie written 
Practice of Fhansacy test may be measuring similar abilities* 
Experience 

Total pharmacy-related experience was found to correlate with none of 
the "theoretical" parts, but did correlate with both parts the "practical" 
portion. Approvable experience also correlated with total ^armacy-related 
experience. Thus, It seons that the "practical" portion of the licensure 
examination measures abilities gained in experiential learning. 
School of Origin 

When correlations of test scores with the school at which the 
candidate received his education were attempted. It was found that 
Washington State University graduates got higher scores in Cbanlstry and 
M&th«natlc8 than did University of Washington graduates. Washington State 
University graduates also listed a greater amount of the 700 hour Intern- 
ship credit for school work for submission to the Board of fhanaacy. Other 
differences were not significant. (See Table 3*) 
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Table 3 



T-Test of Differences Due to School of Origin 






WSU Graduates 


Graduates 










Mean 




Sianlfican 




(naJO) 




t Value 


(P«) 












Am4 w 




75.8 


a.iS 


0.05lf* 




87.6 




2.30 


0.02U* 






71.6 


1.07 


0.29 


in8>riD8icoi>ogy 




71.0 


1.14 


0.26 






81*2 


»0eQ5 


0.96 








1.39 


0.17 


Experience 










Total Ituunnacy 


2.10 


2.54 


-1.30 


0.20 


Approvable 


2.00 


1.62 


1.95 


0.055 


Concurrent 300 bra. 


Jf.50 


5.79 


2.37 


0.020* 


Academic 










Cuaailatlve Grade 






1.2U 


0.22 


Point Aversige 


2.8U 


2.71 



*p<D.05 



It is dangerous to speculate as to the causes of the differences in 
test scores and to attempt to relate them to the educational process in 
different schools. The differences found, therefore, are siapOy reported 
without coment. 
Eactemahip Program 

Candidates were asked tdxether they had participated in an extemshlp 
program. Several WSU graduates thoufijit that they had, therefore it was 
necessary to check the records and separate those who had completed the 
UW rotating extemshlp being Investigated. Testing these variables for 
the entire group, we found that former externa did not perform as well as 
non-extems on Chemistry, Mathematics, and Written Practical, but the 
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externship graduates bad significantly less total pharmacy- related 
experience than non-extemship graduates. Further, the academic grade 
point averages were not different for the tvo groups. Since that sampling 
procedure included WSU graduates tvho had better test scores in certain 
subjects than IM graduates, the tests were again done for a sample of only 
W graduates^ For that U^-^* group, there were found no significant differ- 
ences between thi; test scores of e:iCtem and non>extem graduates. (See 
T&ble 4. ) There was a significant difference between the total pharmacy 
related experience of the two groups. Since the UW externship program 
was elective, it appears that inexperienced students found the p^gram 
more attractive. On the other hand, students with a large amount of 
phan&acy experience apparently perceived less need, or they preferred to 
Keep the Jobs they had rather than to be involved £\ill-time in this type 
of learning experience. 



Table h 

T-Test of Differences JDue to Sxtemship 
(IM Graduates Obly) 



Externship Non-Extemshlp 
Graduates Graduates 



lAcensing Examination 
Cb«aistry 
Mathematics 
Fbarmacy 
Fhansaeology 
Srrors/cmlssions 
Written Practical 

Experience 

Total Fhaxnacy 
Approvable 

Academic 

Cumulative Grade 
Boint Average 



Mean 
(n«l8) 



79.5 
71.0 

69.9 
82.0 

72.8 

1.88 
1.65 

2^67 



Mean 
(n«50) 



2.60 
1.60 



2.72 



t Value 



Significance 
(p«) 



76.7 


-1.65 


0.11 


85.8 


-l.UO 


0.17 


71.9 




0.66 


75.1 


-1.50 


0.1^^ 


80.7 


0.62 


0.54 


75.5 


.1.38 


0.18 



-5.17 

0.20 



.0.1*8 



0.00* 
0.8U 



0.63 
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It is interesting to relate these results to a study performed in 1972 
by one of the authors (4). In that study, the laboratory class in Pre- 
scription Practice was divided into two groups. One randomly selected 
group received instruction in the school laboratory^ and one received 
instruction In community or hospital dispensaries by pharmacists following 
specific school guidelines. These two groups of students did not perform 
differently on either a pre-experience or post-eacperience test of drug 
product imovledge. However, the stud^ts in both groups who had more than 
100 hours of phprmacy experience did significantly better on the tests 
t-han those having less* The conclusion fxm that study was that expe* 
rience, whether school-controlled or not, contributed sisaificantly to a 
studsit's knowledge of drug products. If we note the lack of test score 
differences in the present report between extern and noQ-extem graduates, 
a similar trend is evident in the two studies. 

Experience Credit and Board of Riarmacy Practices 

Two questions on the Supplementary Personal Data sheet were designed 
to discover the magnitude of the problem of being able to count only a 
part of all pharmacy experience that a candidate bad. The differences 
between total pharmacy-related experience and that judged by the candidate 
as approvable by the Board were studied. Among the candidates, S% re- 
ceived credit for all the pharmacy experience they had and 37% had phar- 
macy experience that could not have been approved by the Board. Thirteen 
percent of the candidates had over 500 hours of pharmacy experience that 
was not approvable. 

One question on the Supplementary Personal Data sheet was asked to 
determine how the maximum of 300 hours internship credit for school courses 
was used. The results categorized as to school of origin are shown in 
Table 5- 
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Table 5 

Candidate Use of 300 Hour Maxintum Concurrent Tiae 



None Used 


13* 




1-50 Hours Used 


6 


3 


^1-100 Hours Used 


21 


0 


101-200 Hours Used 


28 


3 


201-300 Hours Used 


55 


87 



SuBBsary 

Through the use of candidate data given at the time of the licoasure 
eicanlnation by the Washington State Bcrard of Riartaacy^ attei&pts were made 
to correlate test scores on the IBVBP standardized examination with them- 
selves, candidatets background, and experience. Of primary interest was 
the effect of the rotating extemship program of the University of 
Washington on the test performance of the candidates. Numerous Interesting 
correlations were found, and speculations as to their meanings were 
discussed. 

The fact that a candidate had completed a rotating extemship was 
not found to affect performance on the various patrts of the examination^ 
however the small numbers of former extems and the fact that their 
characteristics were not great3y different from other candidates made any 
conclusions tentative. 

Experience gained with this kind of study can be of assistance in 
developsent of evaluative methods for esgperiential learning in pharmacy 
and to give indications of suitability of the NABP standardised examina- 
tion in measuring coiapetence. Further studies are desirable for both 
the Boards and Colleges of Pharmacy- 
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